52 research outputs found
Automated Video Game Testing Using Synthetic and Human-Like Agents
In this paper, we present a new methodology that employs tester agents to
automate video game testing. We introduce two types of agents -synthetic and
human-like- and two distinct approaches to create them. Our agents are derived
from Reinforcement Learning (RL) and Monte Carlo Tree Search (MCTS) agents, but
focus on finding defects. The synthetic agent uses test goals generated from
game scenarios, and these goals are further modified to examine the effects of
unintended game transitions. The human-like agent uses test goals extracted by
our proposed multiple greedy-policy inverse reinforcement learning (MGP-IRL)
algorithm from tester trajectories. MGPIRL captures multiple policies executed
by human testers. These testers' aims are finding defects while interacting
with the game to break it, which is considerably different from game playing.
We present interaction states to model such interactions. We use our agents to
produce test sequences, run the game with these sequences, and check the game
for each run with an automated test oracle. We analyze the proposed method in
two parts: we compare the success of human-like and synthetic agents in bug
finding, and we evaluate the similarity between humanlike agents and human
testers. We collected 427 trajectories from human testers using the General
Video Game Artificial Intelligence (GVG-AI) framework and created three games
with 12 levels that contain 45 bugs. Our experiments reveal that human-like and
synthetic agents compete with human testers' bug finding performances.
Moreover, we show that MGP-IRL increases the human-likeness of agents while
improving the bug finding performance
Enhancing the Monte Carlo Tree Search Algorithm for Video Game Testing
In this paper, we study the effects of several Monte Carlo Tree Search (MCTS)
modifications for video game testing. Although MCTS modifications are highly
studied in game playing, their impacts on finding bugs are blank. We focused on
bug finding in our previous study where we introduced synthetic and human-like
test goals and we used these test goals in Sarsa and MCTS agents to find bugs.
In this study, we extend the MCTS agent with several modifications for game
testing purposes. Furthermore, we present a novel tree reuse strategy. We
experiment with these modifications by testing them on three testbed games,
four levels each, that contain 45 bugs in total. We use the General Video Game
Artificial Intelligence (GVG-AI) framework to create the testbed games and
collect 427 human tester trajectories using the GVG-AI framework. We analyze
the proposed modifications in three parts: we evaluate their effects on bug
finding performances of agents, we measure their success under two different
computational budgets, and we assess their effects on human-likeness of the
human-like agent. Our results show that MCTS modifications improve the bug
finding performance of the agents
Highly Dependable Concurrent Programming Using Design for Verification
There has been significant progress in automated verification techniques based on model checking. However, scalable software model checking remains a challenging problem. We believe that this problem can be addressed using a design for verification approach based on design patterns that facilitate scalable automated verification. In this paper, we present a design for verification approach for highly dependable concurrent programming using a design pattern for concurrency controllers. A concurrency controller class consists of a set of guarded commands defining a synchronization policy, and a stateful interface describing the correct usage of the synchronization policy. We present an assume-guarantee style modular verification strategy which separates the verification of the controller behavior from the verification of the conformance to its interface. This allows us to execute the interface and behavior verification tasks separately using specialized verification techniques. We present a case study demonstrating the effectiveness of the presented approach
Automatic Detection of Shared Objects in Multithreaded Java Programs
This paper presents a simple and efficient automated tool called DoSSO that detects shared objects in multithreaded Java programs. Our main goal is to help programmers see all potentially shared objects that may cause some complications at runtime. This way programmers can implement a concurrent software without considering synchronization issues and then use appropriate locking mechanism based on the DoSSO results. To illustrate the effectiveness of our tool, we have petformed an experiment on a multithreaded system with graphical user intetfaces and remote method invocations and achieved promising results
An Assume Guarantee Verification Methodology for Aspect-Oriented Programming
We propose a modular verification methodology for aspect oriented programs. Aspects are the new modularization units to encapsulate crosscutting concerns and have powerful features whose effects can drastically change software behavior. Having such an impact on behavior requires an automated verification support. In this work we introduce a technique that separately answers two questions: "does the aspect have the provisioned effect?" and "does the base program satisfy the assumptions of the aspect?" To answer these questions modularly, we propose using design contracts and state machines as aspect interfaces. An aspect interface both closes the environment of the aspect and specifies its assumptions on any base code. We show that our approach can be used in verifying AspectJ programs modularly and checking compatibility for aspect reuse
Highly dependable concurrent programming using design for verification
There has been significant progress in automated verification techniques based on model checking. However, scalable software model checking remains a challenging problem. We believe that this problem can be addressed using a design for verification approach based on design patterns that facilitate scalable automated verification. In this paper, we present a design for verification approach for highly dependable concurrent programming using a design pattern for concurrency controllers. A concurrency controller class consists of a set of guarded commands defining a synchronization policy, and a stateful interface describing the correct usage of the synchronization policy. We present an assume-guarantee style modular verification strategy which separates the verification of the controller behavior from the verification of the conformance to its interface. This allows us to execute the interface and behavior verification tasks separately using specialized verification techniques. We present a case study demonstrating the effectiveness of the presented approach
Scalable software model checking using design for verification
Abstract. There has been significant progress in automated verification techniques based on model checking. However, scalable software model checking remains a challenging problem. We believe that this problem can be addressed using a design for verification approach based on design patterns that facilitate scalable automated verification. We have been investigating a design for verification approach based on the following principles: 1) use of stateful, behavioral interfaces which isolate the behavior and enable modular verification, 2) an assume-guarantee style verification strategy which separates verification of the behavior from the verification of the conformance to the interface specifications, 3) a general model checking technique for interface verification, and 4) domain specific and specialized verification techniques for behavior verification. So far we have applied this approach to verification of synchronization operations in concurrent programs and to verification of interactions among multiple peers in composite web services. The case studies we conducted indicate that scalable software verification is achievable in these application domains using our design for verification approach.
- …